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(57) ABSTRACT 

A system for transcoding multiple channels of compressed 
video streams using a self contained data unit such as an 
autonomous frame includes an autonomous frame process- 
ing unit having an autonomous frame generator and an 
autonomous frame recoder. The autonomous frame genera- 
tor receives video data and divides it into a series of 
autonomous frames. Each autonomous frame preferably 
comprises 1) a frame header including all header informa- 
tion from the original video data plus enough additional 
information to allow the frame to be recoded using pre- 
defined autonomous frame structure, and 2) a frame payload 
including the original video data information. The autono- 
mous frame recoder process the autonomous frames includ- 
ing extracting processing parameters, extracting the video 
data and setting up or initializing the recoder to process the 
extracted video data. The autonomous frame recoder pref- 
erably further comprises a parser coupled to an initialization 
unit and a recoder. The present invention also includes a 
method for processing video data including the steps of: 
receiving a video bitstream, storing recoding information, 
dividing the video bitstream into a plurality of autonomous 
frames each frame including a portion of the video bitstream 
and recoding information, outputting the plurality of autono- 
mous frames, receiving the plurality of autonomous frames, 
extracting processing information from the autonomous 
frame, extracting video data from the autonomous frame, 
setting the recoding according to the processing information 
and recoding the extracted video data. 
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SYSTEM AND METHOD FOR require excessive amounts of time, compression is an 

TRANSCODING MULTIPLE CHANNELS OF approach that has been used to make digital video images 

COMPRESSED VIDEO STREAMS USING A more transportable. Digital video compression schemes 

SELF-CONTAINED DATA UNIT allow digitized video frames to be represented digitally in 

5 much more efficient manner. Compression of digital video 

This Application claims the benefit of Sen No. 60/094, l [ P T&C ^ to , transmit the compressed signal by 

364 filed Jul 27 1998 digital channels at a fraction of the bandwidth required to 

' ' transmit the original signal without compression. Intema- 

BACKGROUND OF THE INVENTION tional standards have been created on video compression 

schemes and include MPEG-1, MPEG-2, H.261, H.262, 

1 . Field of the Invention 10 h.263, etc. These standardized compression schemes mostly 
The present invention relates generally to systems and rely on several key algorithm schemes: motion compensated 

methods for processing compressed bitstreams of data. In transform coding (for example, DCT transforms or wavelet/ 
particular, the present invention relates to a system and a sub-band transforms), quantization of the transform 
method for transcoding video bitstreams on frame and coefficients, and variable length encoding (VLC). The 
smaller basis. Still more particularly, the present invention 15 motion compensated encoding removes the temporally 
relates to a system and method for transcoding multiple redundant information inherent in video sequences. The 
channels of compressed video streams using self-contained transform coding enables orthogonal spatial frequency rep- 
data units such as autonomous frames wherein such an resentation of spatial domain video signals. Quantization of 
autonomous frame includes a frame header portion and a transformed coefficients reduces the number of levels 
frame payload portion. 20 T f^ mx ^ to represent a given digitized video sample and is 
r .. the major factor in bit usage reduction in the compression 

2. Description of the Related Art process. The other factor contributing to the compression is 
There are presently a variety of different communication the use of variable length coding (VLC) so that most 

channels for transmitting or transporting video data. For frequently used symbols are represented by the shortest code 

example, communication channels such as digital subscriber ^ word. In general, the number of bits used to represent a 

loop (DSL) access networks, ATM networks, satellite, or given image determines the quality of the decoded picture, 

wireless digital transmission facilities are all well known. The more bits used to represent a given image, the better the 

The present invention relates to such communication image quality. The system that is used to compress digitized 

channels, and for the purposes of the present application a video sequence using the above-described schemes is called 

channel is defined broadly as a connection facility to convey 30 an encoder or encoding system. 

properly formatted digital information from one point to In the prior art compression schemes, the quantization 

another. A channel includes some or all of the following scheme is lossy, or irreversible process. Specifically, it 

elements: 1) physical devices that generate and receive the results in loss of video textural information that cannot be 

signals (modulator/demodulator); 2) physical medium that recovered by further processing at a later stage. In addition, 

carries the actual signals; 3) mathematical schemes used to 35 the quantization process has direct effect on the resulting bit 

encode and decode the signals; 4) proper communication usage and decoded video quality of the compressed bit 

protocols used to establish, maintain and manage the con- stream. The schemes at which the quantization parameters 

nection created by the channel. The concept of a channel are adjusted control the resulting bit rate of the compressed 

includes but is not limited to physical channel, but also bit stream. The resulting bit stream can have either a 

logical connections established on top of different network 40 constant bit rate (CBR) or a variable bit rate (VBR). CBR 

protocols, such as xDSL, ATM, wireless, HFC, coaxial compressed bit stream can be transmitted over channel 

cable, etc. delivers digital information at a constant bit rate. 

The channel is used to transport a bitstream, or a con- A compressed video bit stream generally is intended for 
tinuous sequence of binary bits used to digitally represent real-time decoded playback at a different time or location, 
compressed video, audio or data. The bit rate is the number 45 The decoded real-time playback must be done at 30 frames 
of bits per second that the channel is able to transport. The per second for NTSC standard video and 25 frames per 
bit error rate is the statistical ratio between the number of second for PAL standard video. This implies that all of the 
bits in error due to transmission and the total number of bits information required to represent a digital picture must be 
transmitted. The channel capacity (or channel bandwidth) is delivered to the destination in time for decoding and display 
the maximum bit rate at which a given channel can convey 50 in timely manner. Therefore, this requires that the channel 
digital information with a bit error rate no more than a given must be capable of making such delivery. From a different 
value. A video channel or video program refers to one or perspective, the transmission channel imposes bit rate con- 
more compressed bit streams that are used to represent the straint on the compressed bit stream. In general, the quan- 
video signal and the associated audio signals. Also included tization in the encoding process is adjusted so that the 
in the video channel are relevant timing, multiplexing and 55 resulting bit rate can be accepted by the transmission chan- 
system information necessary for a decoder to decode and nel. 

correctly present the decoded video and audio signals to the Because both temporal and spatial redundancies are 
viewer in a time continuous and synchronous manner. And removed by the compression schemes and because of van- 
finally, a multiplex is a scheme used to combine bit stream able length encoding, the resulting bit stream is much more 
representations of different signals, such as audio, video, or $0 sensitive to bit errors or bit losses in the transmission process 
data, into a single bit stream representation. than if the uncompressed video is transmitted. In other 
One problem with existing communication channels is words, minor bit error or loss of data in compressed bit 
their ability to handle the transportation of video data. Video stream typically results in major loss of video quality or even 
data is much larger than many other types of data, and complete shutdown of operation of the digital receiver/ 
therefore, requires much more bandwidth from the commu- 65 decoder. 

nication channels. Since transmission of digitally sampled Further, a real-time multimedia bit stream is highly sen- 

video data with existing communication channels would sitive to delays. A compressed video bit stream, when 



11/19/2003, EAST Version: 1.4.1 



US 6,483,543 Bl 

3 4 

transmitted under excessive and jittery delays, will cause the bitstream, storing recoding information, dividing the video 

real-time decoder buffer to under flow or overflow, causing bitstream into a plurality of autonomous frames each frame 

the decoded video sequence to be jerky, or causing the audio including a portion of the video bitstream and recoding 

video signals out of synchronization. Another consequence information, outputting the plurality of autonomous frames, 

of the real-time nature of compressed video decoding is that s receiving the plurality of autonomous frames, extracting 

lost compressed data will not be re-transmitted. processing information from the autonomous frame, extract - 

Despite the increase in channel bandwidth, there contin- ing video data from the autonomous frame, setting the 

ues to be a need for adjusting the number of bits for recoding according to the processing information and rccod- 

representing a bitstream to the amount of available channel ing the extracted video data, 

bandwidth. Another particular problem, especially when 10 

several channels are multiplexed over a single channel, is the ' BRIEF DESCRIPTION OF THE DRAWINGS 

allocation of the available bandwidth to multiple channels. fig. 1 is a block diagram of a transcoding system 

Often it is necessary to recode bitstreams to maximize the mchldiDg the amon0 mous frame processing unit of the 

utilization of the chaonel bandwidth. However, the use of presen t invention, 

compression techniques also introduces significant compu- 15 ' 

tationaJ complexity into botb the encoding and decoding ™ f Z » » Wo * dia 8 ram ° f a fiisl P»ft™I embodi- 

process. Specifically, the compressed video bit streams, at ment <he autonomous frame processmg unit of the 

any given bit rate, cannot be altered again to a different bit P resen tnven on. 

rate without decoding and recoding. In addition, the result- FIG. 3 is a block diagram of a first embodiment for the 

ing number of bits required to represent digital video pic- 20 autonomous frame generator of the present invention, 

tures varies from picture to picture and the coded pictures FIG. 4 is a block diagram of a first embodiment for the 

are highly correlated as a result of motion estimation. The autonomous frame recoder of the present invention, 

problem of delivering real-time digital video bit stream over FIG. 5 is a block diagram of an exemplary embodiment 

a channel of a given bandwidth becomes even more complex f or the recoder of FIG. 4. 

because the available bandwidth must be matched to the 25 FIG fi fa a U(Jck of a secQQd embodiment for the 

coded video bit stream rate. When the mismatch occurs, autonomous frame processing unit of the present invention, 

recoding, or re-compression, must be done. A final problem y ^ „ . ,...1 ri 

• *i_ 4 • *• j ■ i II j- FIG. 7 is a block diagram ot a third embodiment tor the 

is that existing receding processes only allow recoding on a r ^a &l um ^ a muu vui^uiuivui ^ l lllv 

.... -I • . „ u i . autonomous frame processing unit of the present invention, 

stream by stream basis. There are many instances when only r 6 * 

a portion of a stream may need to be recoded to resolve a 30 FIG. 8 is a block diagram illustrating several exemplary 

temporary shortage of channel bandwidth. embodiments for an autonomous frame based on an access 

Therefore, there is a need for a system and method for unil ( AU )> a s P Uce aod a macro block - 

transcoding a bitstream on a frame or smaller basis. FIG. 9 is a block diagram of a fourth embodiment for the 

Furthermore, there is a need for a system that allows autonomous frame processing unit of the present invention, 

transcoding on a section of the compressed video data and 35 FIG. 10 is a block diagram of a fifth embodiment for the 

on an autonomous basis anywhere within a video stream. autonomous frame processing unit of the present invention. 

SUMMARY OF THE INVENTION FIG. 11 is a block diagram of a sixth embodiment for the 

„, . . .. , - . j autonomous frame processing unit of the present invention. 

The present invention overcomes the deficiencies and r & r 

limitations of the prior art with a system and a method for 40 FIG. 12 is a block diagram of a first statistical 
transcoding multiple channels of compressed video streams re-multiplexer using autonomous frame processing accord- 
using autonomous frames. More particularly, a system in S t0 me present invention. 

according to the present invention includes an autonomous FIG. 13 is a block diagram of a second statistical 

frame processing unit having an autonomous frame genera- re -multiplexer using autonomous frame processing accord - 

tor and an autonomous frame recoder. The autonomous 45 ing to the present invention, 

frame generator receives video data and divides it into a nc-rAir nconDiDnnM nu tuc 

series of autonomous frames. Each autonomous frame pref- D£ J^2^^}^^r^ 

erably comprises 1) a frame header including all header PREFERRED EMBODIMENT 

information from the original video data plus enough addi- While the present invention will now be described with 

tional information to allow the frame to be recoded using 50 particularity for the handling of MPEG-2 digital video 

pre-defined autonomous frame structure, and 2) a frame compression, those skilled in the art will recognize that the 

payload including the original video data information. The principles of the present invention may be applied to a 

autonomous frame generator outputs autonomous frames to variety of other related video compression schemes such as 

the autonomous frame recoder which in turn process the the H.26X video conference signals. Specifically, the present 

autonomous frame including extracting processing 55 applications discloses a technique of adding and processing 

parameters, extracting the video data and setting up or autonomous frames that can be used to selectively perform 

initializing the recoder to process the extracted video data. recoding. Furthermore, those skilled in the art will recognize 

The autonomous frame recoder preferably further comprises that the bit streams discussed below are unscrambled. This 

a parser coupled to an initialization unit and a recoder. The allows direct access to all the bitstream content for rate 

autonomous frame recoder outputs a bitstream that has been <so reduction, recoding, and timing re-stamping. In cases where 

adjusted in the number of bits used to represent the data. In the bitstreams are scrambled, they first need to be 

other embodiments, the autonomous frame processing unit unscrambled, spliced and then re -scrambled, 

may include a plurality of autonomous frame generators and System Overview 

a plurality of autonomous frame recoders coupled in a Referring now to FIG. 1, a preferred embodiment of a 

variety of different configurations. system 100 constructed according to the present invention 

The present invention also includes a method for process- and including an autonomous frame processing unit 108 is 

ing video data including the steps of: receiving a video shown. The system 100 is adapted for use on an MPEG-2 
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transport stream. The preferred embodiment of the system the following sections, the present invention is disclosed in 

100 preferably comprises a separation unit 102, a the context of receding of MPEG-2 bit streams, whether it 

de-multiplexer 104, an elementary stream bit parser 106, the s is transport stream, program stream, PES stream or elemen- 

autonomous frame processing unit 108, a stream bit multi- tary stream. This section provides brief overview of the 

plexer 114, and a transport multiplexer 116. s MPEG-2 bit stream syntax for convenience and ease of 

The separation unit 102 has an input coupled to line 120 understanding of the concept in the present invention, 
and receives an MPEG-2 transport stream. The separation MPEG-2 compression standard consists of two layers of 
unit 102 preferably de-multiplexes the system layer stream, information: the video or audio layer and the system layer, 
removing filler packets/frames as appropriate, to obtain the The video layer includes the elementary stream (ES) layer, 
video bit stream, the audio bit stream and a data bit stream. 10 This layer defines how compressed video (or audio) signals 
The video bit stream is provided on a first output, the audio are sampled, motion compensated, transform coded, 
bit stream is provided on a second output, and a data bit quantized, and represented by different variable length cod- 
stream is provided on a third output. Those skilled in the art ing (VLQ tables. The recoding (or transcoding) of pre- 
will understand the operation of the separation unit 102, compressed MPEG-2 bit streams is a process in which the 
based on the protocol being used for the bit stream. Based 15 bit stream signal is redefined in this layer, 
on the protocol, the bit stream can be divided into the xh e system layer is defined to allow the MPEG-2 decoder 
sub-streams according to the present invention. The audio to correctly decode audio and video signals, present the 
stream and data stream are output directly to the transport decoded result to the video screen in a time continuous 
multiplexer 116 which recombines these streams with the manner. The system layer also includes provisions that allow 
recoded or processed video. Even though a compressed bit 20 unambiguous multiplexing and separate of audio and video 
stream typically includes a multiplex of compressed audio, compressed signals, and different channels of audio and 
video and auxiliary data bit streams, recoding process in the video compressed signals. The system layer consists of two 
present invention refers only to the process applied to sub-layers. The first layer is the PES layer, this layer defines 
compressed video bit stream. how the ES layer bit stream is encapsulated into variable 

The de-multiplexer 104 is coupled to receive the video 25 length packets, called PES packets. In addition, presentation 

stream from the separation unit 102. The de-multiplexer 104 m <i decoding time stamps (PTS/DTS) are added to the PES 

extracts a video elementary stream payload from the video packets. There are two different sub-layers above the PES 

transport stream and in turn sends the video elementary i a y er ^ me transport layer and program system layer, 

stream payload to the elementary stream bit parser 106 The' ^ t n , defines how the p£s ^ are 

elementary stream bit parser 106 receives the output of the 30 ketized into ^ sized t rt ket of 188 

de-multiplexer 104 and divides it into access units that can bytes>AdditionaJ timing information and multiplexing infor- 

include one or more of a transform coefficient component, a matkm are added to me lr rt k ^ rcsultin stfcam 

motion vector component, and an auxiliary mformation of Qrt keU ^ caUed Qrt stream x ort 

component. The access units are provided on hue 122 and stream ^ for ^ ^ cnvironmcnts whcrc errors are 

sent to the autonomous frame processmg unit 108. 35 ^ ^ as s Qr ^ lossy or noisy 

The autonomous frame pro^ mcdia T icd Hcation of fr rt stream inchldcs 

an input and output. Hie input of the autonomous frame Difect Broadcast Service (DBS)j ^ ital or caWe 

processing unit 108 is coupled to the output of the elemen- &ervice ^ 5road5and transmission systems, etc. 

tary stream bit parser 106 to receive access units. The _ . . 

autonomous frame processing unit 108 receives video data 40 ™ e P^gram system layer defines how the PES packets 
and divides it into a series of autonomous frames and are encapsulated into variable size packets^ Additional Urn- 
performs recoding on the autonomous frames. Each autono- ** and multiplexing information are added to the program 
mous frame preferably comprises 1) a frame header includ- s Vf tem }^ ™ e P' 0 ?** stream * desi ^ f ° r ^ ™ 
ing picture header information from the original video data relaUvely error-free envn-onments and is suitable for apph- 
plus enough additional information to allow the frame to be 45 ca ' 10ns lhat mav involve software processing of system 
recoded, and 2) a frame payload including the original video ^formation such as interactive multimedia applicaUons^A 
data picture information. The autonomous frame processing ty f^ al I , ap 1 ph ^° f p [°S ram strcam mchldcs D ^ tal 
unit 108 recedes the video data on an autonomous frame to satlle Disks < DVD > and vldeo seivere * 
generate and video transport stream. The output of the In general a video bit stream can be in elementary stream 
autonomous frame processing unit 108 is coupled by line 50 (ES) format, which means that no PES, transport or program 
124 to the stream bit multiplexer 114. system layer information is added to the bit stream. The 
The stream bit multiplexer 114 essentially performs an v i dc <> bit stream can also be represented in the form of PES 
inverse function of the de-multiplexer 104, and combines stream, transport stream or program stream. For a given 
the output of the autonomous frame processing unit 108 with video bit stream, the difference between these different bit 
other information to re-form the video bitstream. The output 55 stream represented in the different layers lies in the timing 
of the stream bit multiplexer 114 is coupled to an input of the information, multiplexing information and other informa- 
transport multiplexer 116 along with the audio and data ^on not directly related to the receding process. The infor- 
signals from the separation unit 102. These signals are then ™\ion required to perform recoding, however, is contained 
combined by the transport multiplexer 116 and output on in the elementary stream layer. 

line 126. Those skilled in the art will recognize from FIG. 1 60 The discussion below on recoding is, therefore, not lim- 

that the present invention can be applied to bit stream based ited to bit streams in any one of the layers. In other words, 

on other transform schemes. the discussion on how to recoding bit streams in one layer, 

MPEG-2 Bit Stream Syntax say in elementary stream layer, can be straightforwardly 

As has been noted above, the present invention is extended to PES stream, transport stream or program 

described in detail with reference to the syntax used in 65 streams as well, by properly taking additional layering 

MPEG-2* but the concept of an autonomous frame applies to information into considerations, even though the elementary 

any number of video transport and compression schemes. In stream layer is the focus as shown in FIG. 1. 
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Autonomous Frame Data Structure 

The present invention advantageously creates a new con- 
cept of an autonomous frame data structure. The autono- 
mous frame is a scheme for partitioning the compressed 
video bit stream such that each unit or autonomous frame, as 
a result of the partitioning, forms a self-contained data unit 
that can be recoded. Below the partitioning of the coded 
video bit stream is defined to achieve this object. While the 
discussion focuses on the coding of MPEG-2 elementary 
stream, the "autonomous frame" concept described here can 
be applied to other compressed video bit streams as well. 

An autonomous frame is a data unit constructed based on 
a subsection of the compressed video bitstream that contains 
all of the information necessary to recode this data unit 
without using any additional information. Specifically, the 
data unit contains all the information required for the 
recoder to initialize itself and operate on the data unit. An 
autonomous frame is defined as consisting of the following 
two key components: a frame header (AFH) and frame 
payload (AFP). The autonomous frame is preferably derived 
from a subsection of the original compressed video 
bitstream, includes an associated recoding method, and has 
the information contained in the frame header of an autono- 
mous frame sufficient to initialize the recoder to perform 
autonomous recoding. It is important to point out that a 
recoding scheme must be defined together with the autono- 
mous frame data structure. For example, an autonomous 
frame, when used as input to a recoder which is not defined 
with the autonomous frame will not necessarily make the 
recoding process autonomous, and vice versa. In other 
words, the recoding method and the type in the autonomous 
frame are coupled and must match for recoding to be 
properly performed. 

Referring now to FIG. 8, one embodiment 802 for the 
autonomous frame adapted for use in processing an MPEG2 
video bit stream is shown. This autonomous frame imple- 
mentation is defined as a data unit of coded picture without 
using the data from other coded pictures. In addition, 
requantization is assumed to be the method of re-encoding. 
For MPEG-2 video bit stream, each autonomous frame is 
advantageously constructed around the boundary of the so 
called access unit (AU) defined according to the MPEG-2 
standard. For MPEG-2, the present invention defines an 
access unit (AU) as all the coded data for a picture, and any 
stuffing that follows it, up to but not including the start of the 
next access unit. If a picture is not preceded by a group_ 
start_code or a sequence_hcader_code, the access unit 
begins with the picture start code or picture_header. If a 
picture is preceded by a group_start_code and/or a 
sequence _header_code, the access unit begins with the first 
byte of the first of these start codes. If it is the last picture 
preceding a sequence_cnd_code in the bit stream all bytes 
between the last byte of the coded picture and the sequence_ 
end_code (including the sequence__end__code) belong to 
the access unit. 

In terms of the syntax of MPEG-2 video bit stream, an 
access unit (AU) is defined in the video elementary stream 
layer 810 and is shown in the following table. 

TABLE 1 



TABLE 1 -continued 



Access Unit Header (AUH) 



sequencc_headcr 
scqueace_ex tension 
8equcnce_uscr_da la 
GOP_header 
user_data 
picturc_header 
picture_extcnsion 
extc nsion__user_data 



Optional 



Optional 



10 



20 



25 



30 



40 



45 



50 



55 



Access Unit Payload (AUP) 
Picture Data 



slice 0x101 



slice OxlXX 



However, it is straightforward to map the access unit infor- 
mation to the upper layer packets, such as PES or transport 
packets. There may be cases when a transport or PES packet 
contains bits from two or more access units, in which case 
only the bits of the access unit under discussion is consid- 
ered and all bits belonging to other access units are ignored. 
For convenience all data bits in an access unit immediately 
before the slice start code 0x101 but after the end of the 
15 picture data are defined as access unit header (AUH) bits. 
The rest of the bits of an access unit are defined to be the 
access unit payload (AUP) bits. The AUH contains infor- 
mation that is used to assist the decoding process and is 
maintained until being updated by the AUH contents from 
the next access unit Some of the information bits in AUH 
must be used to perform the recoding process and thus must 
be available to the recoder. 

For MPEG-2, the present invention defines the self con- 
tained data structure, called an autonomous frame, to include 
a frame header (AFH) and frame payload (AFP), as noted 
above. The autonomous frame header (AFH) includes an 
updated version of AUH and additional auxiliary informa- 
tion used to assist the recoding process. For example, such 
auxiliary information preferably includes a bit budget per 
picture (target bit usage of x bits), presentation and display 
time stamps, length of the frame in byte or packet, etc. The 
auxiliary information bits include all the information to 
perform recoding of the access unit. In other words, the AFH 
can be considered list of all global variables and their values 
that may be needed for recoding. Again, it is important to 
point out that the particular recoding technique described in 
FIG, 4 for recoder 408 is assumed to be requantization. This 
approach advantageously does not require the use of refer- 
ence coded pictures. 

The autonomous frame payload (AFP) is preferably the 
same as the AUP, but may be different. The AUP does not 
exclude the scenario when all elementary stream bits are 
contained as part of the PES or transport packets. In this 
case, all information contained in the transport packet header 
fields are not considered part of AUP It is important to point 
out that AFP described above may be interpreted as other 
than a copy of the AUP. When the access unit is in pack- 
etized form of the upper layer syntax, the associated upper 
layer system information may also be carried forward into 
the autonomous frame payload. To make the concept easier 
to explain, we limit our discussion to the access unit in its 
elementary stream form, and the assumption that the AUP is 
the same as the AFP. 

The relationship of the autonomous frame to the access 
unit is best illustrated in FIG. 8. In order the construct the 
autonomous frame 802, a parsing process is used to obtain 
the frame header. Specifically, the auxiliary information 
contained in the frame header is derived not from the latest 
access unit only, but from all of the access unit headers of 
all previous access units, up to and including the current 
access unit. A processing unit, called the prc-parser, must be 
used to generate the autonomous frame as will be discussed 
below. The pre-parser extracts the AUH information col- 
lected so far from all of the access units and uses it to update 
and set the proper fields in the auxiliary information data 
section of the autonomous frame. Once the AFH is created 
the AFP which is the video data from the original bit stream 
is added to create the autonomous frame 802. 



60 



65 
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Those skilled in the art will recognize that the autono- generator 302. The access unit generator 302 receives the 

mous frame concept is not limited to a mapping between packets at the transport layer and converts them to access 

access units and autonomous frames, but could occur at a units as described above with reference to FIG. 8. In an 

variety of other levels to provide any level of partitioning of exemplary embodiment, this could be the same as 

a video bitstream. TWo alternate embodiments 804, 806 for 5 de-multiplexer 104. The packets on the transport layer are 

the autonomous frame arc also shown in FIG. 8. The other divided into a transport header and a transport payload to 

embodiments 804, 806 of the autonomous frame define the form a unit of convenient size for transmission over the 

frame boundary to coincide with smaller data units within an channel. The packets on the transport layer arc received by 

access unit. For example, the autonomous frame 804 has an the access unit generator 302 and reformatted into access 

AFH that includes the auxiliary information, but information 10 units including an AUH and AUP. The access unit generator 

from the AUH and the slice header as graphically depicted. 302 uses the one or more transport headers to reformat the 

The AFP for the autonomous frame 804 includes only a transport payloads into AU having an AUH and AUP, and 

portion of the data from the AUP, namely, the slice payload. then outputs the AU. The transport headers are discarded 

A similar embodiment on a finer level of granularity is once the AUs have been generated, 

shown as autonomous frame 806 where the AFH includes 15 The output of the access unit generator 302 is provided as 

the auxiliary information, information from the AUH, the an input to both the header extraction unit 304 and the 

slice header and the macro block header, and the AFP payload transfer unit 310. The payload transfer unit 310 is 

includes the macro block payload. The autonomous frame similar to the header extraction unit 304, but processes the 

may be alternatively defined to be one autonomous frame AUs to generate the AUP. In the simplest embodiment, the 

per multiple macro blocks, one autonomous frame for any 20 payload transfer unit 310 receives an AU and identifies the 

contiguous section of a coded picture, or a variety of other picture data (as shown in Table 1) and outputs the picture 

embodiments for any desired level of granularity, along with data as the frame payload. For the case of MPEG-2 video 

the associated appropriate recoding scheme. bitstream, the frame payload is preferably a subsection of the 

Autonomous Frame Processing Unit MPEG-2 video bitstream, in the form of either elementary 

Referring now to FIG. 2, the first and preferred embodi- 25 stream, PES stream, transport stream or program stream, 

ment for the autonomous frame processing unit 108a of the The output of the payload transfer unit 310 is coupled to an 

present invention is shown in more detail. The autonomous input of the autonomous frame construction unit 312 to 

frame processing unit 108cpreferably comprises an autono- provide the AUPs for joining with the AFH to form an 

mous frame generator 202 and an autonomous frame recoder autonomous frame. 

204. The autonomous frame generator 202 receives video 30 The header extraction unit 304 receives the AU and 

data and divides it into a series of autonomous frames 802 extracts the AUH from the AU. The header extraction unit 

as described above with reference to FIG, 8. The autono- 304 acquires the data from the AUH that is used to create the 

mous frame generator 202 produces autonomous frames 802 AFH. The AUH for MPEG-2 may have one of three formats, 

from the input video signal and outputs the frames to the First, the AUH may include only the picture„header, the 

autonomous frame recoder 204 via line 220. The autono- 35 picture_extension and the extension__user_data. In such 

mous frame generator 202 will be described in more detail case, the header extraction unit 304 determines these three 

with reference to FIG. 3. The autonomous frame recoder 204 values from the AUH and outputs them. Second, the AUH 

in turn processes the autonomous frames 802 including may include the GOP_Jieader, user__data, picture_header, 

extracting processing parameters, extracting the video data the picture_extension and the extension__user_data. In this 

and setting up or initializing a recoder to process the 40 second case, the header extraction unit 304 determines these 

extracted video data. The autonomous frame recoder 204 five values from the AUH and outputs them. Finally, the 

will be described in more detail with reference to FIG. 4. The AUH may include a sequence_header, a sequence_ 

autonomous frame recoder 204 outputs a bitstream that has extension, sequence_user_data, the GOP_header, user_ 

been adjusted in the number of bits used to represent the data data, picture_header, the picture_extension and the 

on line 124. 45 extension_user_data. In this case all the values are deter- 

Referring now to FIG, 3, a preferred embodiment for the mined and output by the header extraction unit 304. These 

autonomous frame generator 202 of the present invention is values will be used in turn by the autonomous frame 

shown. The autonomous frame generator 202 preferably construction unit 312 to create the autonomous frame. Thus, 

comprises an access unit generator 302, a header extraction the header extraction unit 304 provides the auxiliary infor- 

unit 304, an auxiliary information updating unit 306, an 50 mation which for the case of MPEG-2 video bit stream, may 

autonomous frame header buffer 308, a payload transfer unit include some or all of the following: a sequence header, a 

310, and an autonomous frame construction unit 312. The sequence extension, an extension and user data, a group of 

autonomous frame generator 202 is particularly advanla- picture header, a picture header and a picture coding exten- 

geous because is receives a video bitstream and converts the sion. 

bitstream into a plurality of autonomous frames that can be 55 The output of the header extraction unit 304 is coupled to 

processed independently by the autonomous frame recoder the input of the auxiliary information updating unit 306. The 

204. For example, the autonomous frames can be processed auxiliary information updating unit 306 also has a data and 

one at a time in sequence as describe with reference to FIG. control outputs coupled to the autonomous frame header 

4 or many at a time as will be described in other embodi- buffer 308. The auxiliary information updating unit 306 

merits. The partitioning of the bitstream into autonomous 60 maintains and keeps the autonomous frame header buffer 

frame that can be independently processed allows the frames 308 current. As additional auxiliary information is extracted 

to be processed in any number of ways, therefore making the from each received AU by the header extraction unit 304, the 

present invention fully scale able and flexible. auxiliary information updating unit 306 stores the new 

The autonomous frame generator 202 has an input information in the autonomous frame header buffer 308. The 

.coupled to line 122 to receive transport packets, for 65 auxiliary information updating unit 306 also receives signals 

example, in MPEG-2 format. The input of autonomous from a recoding algorithm controller/selector indicating the 

frame generator 202 is formed by the input of the access unit recoding method used, the desired bit budget for this picture 
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and other information not available from the current or AFHs and only process the AFPs. The recoder 408 then 
previous AUH, but needed for receding. The auxiliary outputs a bitstream which may be an elementary stream, 
information updating unit 306 is also responsible for man- packetized elementary stream (PES), transport, or an 
aging the data in autonomous frame header buffer 308 since autonomous frame. The controller 402 is coupled to control 
there may be auxiliary information for a number of streams 5 the parser 404, the initialization unit 406 and the recoder 408 
stored in the autonomous frame header buffer 308. to coordinate and control the operation of the above men- 
Trie autonomous frame header buffer 308 is any one of a tioned components. For example, the recoder 408 cannot 
variety of conventional buffers and is used to store the data start recoding until the parser 404 has completed parsing by 
necessary for re-encoding any given picture. The autono- extracting and providing the auxiliary information and the 
mous frame header buffer 308 is preferably sized to hold the 10 initialization unit 406 has properly set the recoder's states, 
auxiliary information for a plurality of bitstreams. The Once the initialization is complete, the controller discon- 
autonomous frame header buffer 308 preferably stores a nects the recoder 408 from the initialization unit 406 and 
current list of values for the sequence _Jieader, a sequence_ performs the recoding using the input on 410. 
extension, sequence_user_data, the GOP_header, user_ The recoder 204 is particularly advantageous because it 
data, picture_header, the picture_extension and the 15 provides autonomous operation by virtue of fact that the 
extension_user_data for each stream. The autonomous recoder 408 can perform the operation completely indepen- 
frame header buffer 308 also stores other auxiliary informa- dently of the processing done on the previous and next 
tion required by a re-encoder to process the AFP. The access units. This capability allows several flexible imple- 
autonomous frame header buffer 308 is coupled to both the mentations of the recoding as will be described in detail 
auxiliary information updating unit 306 and the autonomous 20 below. Again, while the present invention is discussed in the 
frame construction unit 312. context of processing of MPEG-2 video elementary streams, 
The autonomous frame construction unit 312 is coupled to the same techniques, with minor modifications, can be 
the autonomous frame header buffer 308 to receive data for applied to the processing of MPEG-2 transport streams, 
generating the AFH and to the payload transfer unit 310 to MPEG-2 PES, MPEG-2 program streams, MPEG-1 streams, 
receive the data used to form the AFP. The autonomous 25 or H.26x based video conferencing signals. Other additional 
frame construction unit 312 basically packages the auxiliary information and protocol data units, such as MPEG-2 trans- 
information retrieved from the autonomous frame header port stream over ATM protocols, layered on top of the above 
buffer 308 into the autonomous frame header (AFH), once signals can also be handled similarly, 
per access unit. All information needed to describe the state For the present invention, recoding is defined in its 
of the recoder for it to perform recoding is retrieved and 30 broadest sense to include partial decoding, recoding, 
stored as the AFH. The autonomous frame construction unit re-quantization, re-transforming, and complete decoding 
312 then transfers the bitstream subsection of the AUP into and recoding. Referring now to FIG. 5, each of these type of 
the AFP. Thus, an autonomous frame is created and output recoding are defined with more particularity. FIG. 5 is used 
on line 220. to show various possibilities for the recoder 408. Some of 
Referring now to FIG. 4, a preferred embodiment for the 35 the elements shown may also be needed for decoding and 
autonomous frame recoder 204 of the present invention is encoding of the video data. Hence in actual implementation, 
shown. The autonomous frame recoder 204 comprises a these common elements may be shared between the recoder 
parser 404, an initialization unit 406 and a recoder 408 408 and the decoder/encoder. Partial decoding refers to path 
operating under the direction of a controller 402. The E where the bitstream is partially decoded, decode system 
autonomous frames are designed to allow autonomous 40 syntax, and video syntax down to the picture header to 
recoding. Specifically, the autonomous frame recoder 204 of perform frame accurate flexible splicing. Recoding refers to 
the present invention is one that can re-encode the access path D where variable length encoding and decoding are 
unit of compressed video elementary stream in an entirely performed and the DCT coefficients may be truncated to 
self-contained manner. In other words, the information con- zero without even going through the inverse quantization 
taincd in the frame header of an autonomous frame is 45 steps. This approach requires the least processing, but in 
sufficient for the system to perform the re-encoding without general causes the greatest amount of quality degradation, 
any external input. The parser 404 has a data input, a data Re-quantization refers to path C where variable length 
output and a control input. The data input of the parser 404 encoding, de-quantization, quantization and decoding are 
is coupled to line 220 to receive autonomous frames from performed but no transform coding is used. The transform 
the autonomous frame generator 202. The parser 404 50 coefficients (DCT coefficients) are requantized before VLC 
receives the autonomous frames, separates the AFH and encoded back. This is the approach preferably used for the 
AFP, and outputs corresponding AFHs and AFPs on line 410 recoder 408. Re-transformation refers to path B where 
such that each portion of the autonomous frame is identified variable length encoding, de-quantization, inverse 
for later processing. The output of the parser 404 is coupled transform, transform coding, quantization and decoding are 
to an input of the initialization unit 406 and a data input of 55 performed. The video frames are constructed without using 
the re-encoder 408. The initialization unit 406 identifies the motion compensation, In the case of B or P pictures, this 
AFH, and extracts the content in the frame header including would mean some of the coded blocks are motion estimated 
the auxiliary information and the AUH. This content is residual errors. Some form of spatial filtering may be used 
output by the initialization unit 406 and input to a control before forward transform coding is used in the encoding 
input of the re-encoder 408. The content is used to initialize 60 process. Recoding refers to path A where the bitstreams are 
the re -encoder 408. The re-encoder 408 is initialized, and as complete decoded to raw video and then encoded including 
a result of this initialization, the re-encoder 408 is set to a the use of motion compensation. Each of the paths A, B, C, 
state just as if it had completed the decoding of the last D, E includes a rate converter for adjusting the rate of the 
access unit prior to this access unit. The recoder 408 also has bitstream to ensure buffer compliance. Each of the rate 
the data input coupled to the output of the parser 404. The 65 converters may be different. For example, the rate converter 
recoder 408 performs the recoding processing using the AUP on path A may be a spatial filter and the rate converter on 
as input. At the data input, the recoder 408 disregards the path C may perform a quantization step size adjustment 
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while the rate converter on path D performs high frequency At this phase, the recoder 612 functions as if it has been 

elimination. Those skilled in the art will also recognize that processing a continuous flow of bitstream up to this point, 

the components of the recoder 408 used (e.g., the path with all parameters needed for the decoding updated, 

through the recoder 408) could also be variably controlled to Therefore, the key function of the controller 614 is to ensure 

provide variable bit rate conversion using the recoder 408. 5 the recoder 612 does not start processing before the header 

In various embodiments, the recoder 408 may include all, parser 606 and the initialization unit 610 have completed 

only some or any combination of these components accord- their processing of the autonomous frame header and have 

ing to which of recoding, re-quantization, re -transforming properly initialized the recoder 612. 

and recoding is performed. Referring now to FIG. 7, a block diagram of a third 

More specifically, the recoder 408 preferably performs 10 embodiment for the autonomous frame processing unit 108c 
only requantization, and thus, follows path C through the of the present invention is shown. The third embodiment of 
rate converter. The inverse transform, motion compensation, the autonomous frame processing unit 108c preferably com- 
motion estimation, transform and additional rate converters prises a pre-parser 602, an autonomous frame creator 604, a 
on paths A, B, D, and E are therefore not required according header parser 606, a payload parser 608, an initialization 
this embodiment. However, in alternate embodiments, the 15 unit 610, and a recoder 612. Many of the components of the 
recoder 408 may perform any level of recoding provided by third embodiment are similar to like components of the 
paths A through E. Generally, motion compensation is most second embodiment, and like reference numerals have been 
computationally expensive, transform coding and inverse used for like terms. The third embodiment for the autono- 
transform coding are also quite expensive. For example, in mous frame processing unit 108c is also similar to the first 
FIG. 5, if we take the path B, motion compensation is 20 embodiment in that the pre-parser 602 and the autonomous 
avoided. If we take path C, both motion compensation and frame creator 604 could be considered to be the autonomous 
transform coding are eliminated. If we take path D, quan- frame generator 202; and the header parser 606, the payload 
tization steps are also eliminated, in addition to motion parser 608, the initialization unit 610, and the recoder 612 
compensation and transform coding. Of course, path A could be considered the autonomous frame recoder 204. The 
performs the entire decoding and encoding processes are is third embodiment for the autonomous frame processing unit 
performed, resulting in the most flexibility and quality 108c also includes a controller 714 that is described in more 
potential, at the cost of being most expensive. detail below. 

Referring now to FIG. 6, a block diagram of a second As shown in FIG. 7, the pre-parser 602 has an input 

embodiment for the autonomous frame processing unit 1086 coupled to line 122 to sequentially receive a plurality of 

of the present invention is shown. The second embodiment 30 access units. The pre-parser 602, however, has a single 

of the autonomous frame processing unit 1086 preferably output coupled to the input of the autonomous frame creator 

comprises a pre-parser 602, a header parser 606, an initial- 604. The pre-parser 602 is used to generate the AFH. The 

ization unit 610, and a recoder 612. The pre-parser 602 has pre-parser 602 extracts the AUH from the input streams on 

an input coupled to line 122 to sequentially receive a line 122 and combines the information in the AUH with 

plurality of access units. The pre-parser 602 has a plurality 35 other auxiliary information to form the AFH. The pre-parser 

of outputs coupled to line 620 and 622, respectively. The 602 then outputs the AFH to the autonomous frame creator 

pre-parser 602 is used to convert access units to autonomous 604. The autonomous frame creator 604 has a second input 

frames. The pre-parser 602 is used to generate a logical coupled to line 122 to receive the access units. The autono- 

autonomous frame and outputs the AFH on line 620 and the mous frame creator 604 also parse the access units, but it 

AFP on line 622. Line 620 is coupled to the header parser 40 extracts the AUP, and uses it to form the AFR The autono- 

606. The header parser 606 extracts the content in the frame mous frame creator 604 then combines the AFP with the 

header including the auxiliary information and the AUH. AFH from the pre-parser 602 to produce autonomous frames 

The header parser 606 outputs the auxiliary information and that are provided at the output of the autonomous frame 

the AUH to the initialization unit 610. The initialization unit creator 604. 

610 applies the auxiliary information and the AUH to the 45 The output of the autonomous frame creator 604 is 

recoder 612 to initialize the recoder 612. Again, the recoder coupled to the input of both the header parser 606 and the 

612 is set to a state just as if it had completed the decoding payload parser 606. The header parser 606 operates as has 

of the last access unit prior to this access unit. The recoder been described above to extract the content in the frame 

612 is a similar type to that disclosed above with reference header including the auxiliary information and the AUH. 

to FIG. 5. 50 The header parser 606 outputs the auxiliary information and 

The controller 614 shown in FIG. 6 coordinates the the AUH to the initialization unit 610. The initialization unit 

operation of the pre-parser 602, the header parser 606, the 610 applies the auxiliary information and the AUH to the 

initialization unit 610, and the recoder 612. At the input of recoder 612 to initialize the recoder 612. Again, the recoder 

each new autonomous frame, the controller 614 first disables 612 is set to a state just as if it had completed the decoding 

the operation of the header parser 606, the initialization unit 55 of the last access unit prior to this access unit. The recoder 

610, and the recoder 612. It then enables the pre-parser 602 612 is a similar type to that disclosed above with reference 

to generate the autonomous frame and separate the frame to FIG. 5. The payload parser 606 also receives the autono- 

into AFH and AFP. Next, the controller 614 enables the mous frames, however, the payload parser 606 extracts the 

header parser 606 to accept the AFH from the pre-parser AFP from each autonomous frame and provides the picture 

602. After that, the controller 614 sends the output of header 60 data of the AUP to the data input of the recoder 612. The 

parser 606 to the initialization unit 610. The controller 614 recoder 612 outputs an elementary bitstream that has been 

then disables the header parser 606 and enables the initial- recoded to the desired bit rate, whether it's CBR or VBR. 

ization unit 610. In this phase, the initialization unit 610 In FIG. 7, the header parser 606, the payload parser 608, 

resets the operation of recoder 612 and initializes it with the the initialization unit 610, and the recoder 612 are controlled 

values obtained from the header parser 606. In the last phase, 65 by controller 714. The controller 714 determines the 

the controller 614 enables the operation of recoder 612 and sequence of operation among all components, similar to the 

disables the header parser 606 and the initialization unit 610. controller 614 in FIG. 6. In this case, the sequence of 
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operation goes as follows. First, the controller 714 disables 
the payload parser 608, the initialization unit 610, and the 
recoder 612 and enables the header parser 606 to parse out 
the AFH. Second, the controller 714 then enables the ini- 
tialization unit 610 to use the parsed AFH to initial values 
used for decoding. Third, the controller 714 enables the 
operation of recoder 612. By this time, the recoder 612 
operates as if it is the middle of the on-going decoding of a 
continuous stream. The controller 714 enables the payload 
parser 608 to allow it to parse out the AFP and output the 
result to the recoder 612. 

Referring now to FIG. 9, a block diagram of a fourth 
embodiment for the autonomous frame processing unit 108ci 
of the present invention is shown. The fourth embodiment 
for the autonomous frame processing unit 108*/ advanta- 
geously provides a divider 902, a plurality of autonomous 
frame generators 904a-904/j, a plurality of autonomous 
frame recoders 906a-906/a and a combiner 908. The fourth 
embodiment for the autonomous frame processing unit 108d 
provides increased processing speed with a parallel archi- 
tecture at the cost of multiple autonomous frame generators 
904a-904* and autonomous frame recoders 906a-906n. 
The divider 902 has an input coupled to line 122 to receive 
access units. The divider 902 sequences or partitions the 
stream of access units received in a serial form and outputs 
a plurality of them in parallel. The divider 902 has a plurality 
of outputs, each output of the divider 902 is coupled to the 
input of a respective autonomous frame generator 
904a-904«. For example, a first output of the divider 902 is 
coupled by line 922 to the input of autonomous frame 
generator 904a, It should be understood that each of the 
autonomous frame generators 904a-904n share a common 
buffer area (not shown) so that all the auxiliary information 
needed for the AFH is available to all other autonomous 
frame generators 904a-904/j, The outputs are coupled by 
other lines 922 to a respective autonomous frame generator 
9046-904/j. Each of the autonomous frame generators 
904fl-904n is preferably similar to that described above with 
reference to FIGS. 2 and 3, and outputs autonomous frames. 
Each of the autonomous frame generators 904a-904/j has an 
output that is coupled to a respective autonomous frame 
recoder 906a-906n via line 924. The autonomous frame 
recoders 906a-906/j are preferably similar to that described 
above with reference to FIGS. 2 and 4. The output of each 
autonomous frame recoder 906a-906n is coupled by line 
926 to a respective input of the combiner 908. The combiner 
908 combines the outputs of multiple autonomous recoders 
906a-906/i into another video bit stream in the same order 
as that of the original bit stream. The output of the combiner 
908 is coupled to and provided on line 124. This architecture 
is advantageous since the access units from the same video 
elementary stream are first partitioned into autonomous 
frame data units and then distributed to different processors 
(autonomous recoders 9 06a -90 Sri) for recoding. This 
scheme allows parallel processing of a single compressed 
video bit stream. 

Referring now to FIG. 10, a block diagram of a fifth 
embodiment for the autonomous frame processing unit 108e 
of the present invention is shown. The fifth embodiment for 
the autonomous frame processing unit 108e is similar to the 
fourth embodiment 108d, however, it provides both parallel 
processing and selective processing. For convenience and 
ease of understanding, like components from FIG. 9 are 
provided with like reference numerals and have a similar 
functionality. The fifth embodiment for the autonomous 
frame processing unit 108e differs from the fourth embodi- 
ment by providing direct transfer lines between one or more 



!3,543 Bl 

16 

of the outputs of the divider 902 and respective inputs of the 
combiner 908. For example, this is illustrated in FIG. 10 by 
lines 1002 and 1004. These direct transfer lines 1002 and 
1004 allow the incoming video bitstrcam to be passed 

5 directly to the combiner 908 without recoding the access 
units transferred along these lines 1002 and 1004. The 
divider 902 provides access units on all its outputs. For the 
outputs connected to the combiner 908 by the direct transfer 
lines 1002 and 1004, the access units are passed to the 

10 combiner 908 unchanged. Thus, this embodiment provides 
selective recoding by virtue of selected access units being 
transferred along the transfer lines 1002 and 1004 while the 
other pass through autonomous frame recoders 906a-906/i. 
The combiner 908 combines the outputs of multiple autono- 

15 mous recoders and the direct transfers into another video bit 
stream in the same order as that of the original bit stream. 
The divider 902 and combiner 908 of FIGS. 9 and 10 operate 
in a coordinated manner. Using FIG. 9 as an example, the 
divider 902 first partitions the incoming (single) bitstream 

20 along access unit boundaries. If the input signal is not in 
elementary stream format, the divider 902 performs the 
equivalent operations on the associated transport packets, 
and in this case, the access units are encapsulated as the 
payload of the corresponding transport packets. 

25 Furthermore, the divider 908 delivers the resulting access 
units to the autonomous frame generators sequentially. The 
order at which 904fl-904n receives the autonomous frames 
can be, for example, in round-robin order or in some other 
fixed order. The combiner 908, correspondingly performs 

30 the time sequential concatenation operation of the resulting 
outputs from the autonomous frame recoders 906a-906/i by 
following the same order as the divider 902. Therefore, some 
form of buffering (although not shown) may also be neces- 
sary between the combiner 908 and the autonomous frame 

35 recoders 906o-906n as will be understood by those of 
ordinary skill in the art. 

FIG. 11 shows a block diagram of a sixth embodiment of 
the autonomous frame processing unit 108/ of the present 
invention. The sixth embodiment of the autonomous frame 

40 processing unit 108/ provides selective recoding of a single 
video bitstream. Selective recoding refers to a process that 
recodes portions of a compressed video. Specifically, in the 
preferred embodiment, one access unit may be recoded, 
while the next one is not recoded. The autonomous frame 

45 processing unit 108/ preferably comprises an autonomous 
frame generator 1102, a first switch 1106, an autonomous 
frame recoder 1104, a second switch 1108, and a by-pass 
path 1110. The autonomous frame generator 1102 has an 
input and an output. The input of the autonomous frame 

50 generator 1102 is coupled to line 122 to receive a serial 
stream of access units. The autonomous frame generator 
1102 is similar to that described above with reference to 
FIGS. 2 and 3, and generates an autonomous frame for each 
access unit. The autonomous frames are output by the 

55 autonomous frame generator 1102 to the input of the first 
switch 1106. The construction of the autonomous frames 
ensures that the recoding can be done independently using 
only the autonomous frame as the input. The first switch 
1106 also has a first output and an second output. The first 

60 output of the first switch 1106 is coupled to the input of the 
autonomous frame recoder 1104 and the second output of 
the first switch 1106 is coupled to the by-pass path 1110. The 
first switch 1106 may be positioned so that it couples the 
output of the autonomous frame generator 1102 to the input 

65 of the autonomous frame recoder 1104, or the output of the 
autonomous frame generator 1102 to the by-pass path 1110. 
Thus, the autonomous frames are either recoded by the 
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autonomous frame recoder 1104 or directly passed to the statistical multiplexing on pre-compressed video signals 
output line 124. The autonomous frame recoder 1104 is without the use of real-time compression systems. The 
similar to that described above with reference to FIGS. 2 and autonomous frame processing can also be used to perform 
4, and recodes the autonomous frames. The output of the statistical re multiplexing. In this case, the statistical remul- 
autonomous frame recoder 1104 is coupled to a first input of 5 tiplexer performs receding, selective or complete recoding, 
the second switch 1108. The second switch 1108 has a pair on all of the participating channels so that the multiplexed 
of inputs and an output, and may be positioned to output output bit stream has a given constant rate. Statistical 
either the signal applied at the first input or the signal applied remultiplexing can be considered as the combination of 
at the second input. The second input of the second switch " selective recoding and channel hopping (described below). 
1108 is coupled to the by-pass path 1110. The output of the 10 On exemplary embodiment for a statistical re-multiplexer 
second switch 1108 is coupled to line 124 to provide the 1200 using autonomous frame processing is shown in FIG. 
selectively recoded bitstream. The first and second switches 12 and comprises a plurality of pre-parsers 1202, a process 
1104 and 1108 are synchronized to be switched at the same allocator 1204, a plurality of autonomous frame recoders 
time to be coupled either the autonomous frame recoder 1206, a re-multiplexer 1208 and a controller 1210. 
1104 or the by-pass path 1110. By selectively switching the 15 The plurality of pre-parsers 1202 are similar to. those 
positions of the first and second switches 1104 and 1108 described above with reference to FIGS. 6 and 7. The 
between bypass or recoding mode, selected autonomous plurality of pre-parsers 1202 are each respectively coupled 
frames may be recoded. The switches 1106 and 1108 are to receive a different video stream being statistically multi- 
controlled by a separate controller, not shown in FIG. 11, but plexed. The plurality of pre-parsers 1202 convert the access 
as will be understood by those of ordinary skill in the art. 20 units (or transport packets or PES packets that contain the 
The controller takes it's decision from algorithms related to access unit) into autonomous frames. If one pre-parser is 
the overall recoding needs of the output channel. For dedicated to the pre-parsing of one channel as preferred, 
example, if the output channel is capable of handling the multiple pre-parsers are needed. Otherwise, the same pre- 
signal without any further recoding, the controller simply parser may be used, on a time division basis, to handle the 
sets both switches to by-pass mode. If the output channel is 25 pre-parsing of multiple channels of video channels. The 
running into severe bandwidth over usage, the controller outputs of the plurality of pre-parsers 1202 are coupled to 
may set the two switches to processing through the autono- respective inputs of the process allocator 1204. The process 
mous frame recoder 1104 all the time. The more general allocator 1204 assigns one of the plurality of autonomous 
situation is that the controller takes the decision to by-pass frame recoders 1206 a given access unit of a given video bit 
or process from a separate rate controller that operates to 30 stream, and its corresponding autonomous frame. The pro- 
optimize the bandwidth utilization of the output channel by cess allocator 1204 determines which autonomous frame is 
recoding some autonomous frames only when bandwidth going to. be processed by which autonomous recoder 1206. 
utilization temporarily exceeds capacity. The process allocator 1204 implements resource sharing 

FIG. 12 is a block diagram of a first statistical algorithms that attempts to achieve optimal quality under 

re-multiplexer 1200 using autonomous frame processing 35 certain constraints. For example, the algorithm may maxi- 

according to the present invention. Statistical multiplexing, mize the number of autonomous frame recoders in use at any 

in general, is a technique that simultaneously encodes mul- time when the bandwidth utilization at the output of the 

tiple channels of video signals and combines the resulting bit re -multiplexer exceeds the channel capacity. The algorithm 

stream into a single bit stream for transmission. Statistical may also allocate the autonomous frame recoders according 

multiplexing explores the variable rate nature of the com- 40 to user-set priorities on different video channels by reducing 

pressed video bit streams and the statistical nature of such bit or by-passing recoding activities on those higher priority 

rates. Specifically, it combines multiple channels of variable (namely requires less recoding) channels. A variety of algo- 

bit rate (VBR) bit streams of compressed video so that the rithms for implementing statistical multiplexing are known 

resulting multiplex has a constant fixed rate. This technique, in the art, any one of which may be used to control the 

when properly implemented, results in significant bandwidth 45 operation of the process allocator 1204. The process allo- 

savings when transmitting multiple channels of compressed cator 1204 has a plurality of outputs each coupled to a 

digital video signals. The statistical multiplexing scheme respective one of the plurality of autonomous frame recoders 

includes the following key components: 1) analysis of video 1206, and for delivering autonomous frames. The plurality 

contents to determine the optimal bit budget allocated to of autonomous frame recoders 1206 arc similar to those that 

each coded picture; 2) coded picture type decision: whether 50 have been described above, and can independently recede 

to use motion compensation; 3) coordinated selection of bit autonomous frames. Each of the plurality of autonomous 

rates across all channels to result in total fixed rate; 4) rate frame recoders 1206 has an output coupled to a respective 

control or coding of pictures in each channel that meets the input of the re -mul tiplexer 1208. The re-multiplexer 1208 is 

bit usage budget determined by the step 3; 5) monitoring and preferably an MPEG-2 transport packet re-multiplexer that 

compensation of timing information on all channels 55 accepts multiple data paths, each carrying a stream of 

(required to carry real-time compressed video signals); 6) transport packets (or PES packets or simply video elemen- 

scheduling and multiplexing of different channels into a tary stream) which contains one compressed video bit 

fixed bandwidth channel. stream and possibly one or more audio or data packets. 

Statistical remultiplexing is a technique that simulta- Finally, a controller 1210 is coupled to the plurality of 

neously recodes multiple channels of video signals and 60 pre-parsers 1202, the process allocator 1204, the plurality of 

combines the resulting bit streams into a single bit stream for autonomous frame recoders 1206, and the re-multiplexer 

transmission. The key difference from statistical 1208. The controller 1210 maintains a database of informa- 

multiplexing, therefore, is that the inputs to the statistical lion from the pre-parser 1202, process allocator 1204, 

remultiplexer are pre-compressed bit streams. The recoding, autonomous recoder 1206 and re-multiplexer 1208. The 

together with the rate control and remultiplexing performs 65 information may include, but is not necessarily limited to, 

the same functions as that of statistical multiplexing. Sta- bit usage per coded picture before and after the recoding, 

tistical remultiplexing allows the system operator to perform state of the processor allocator, buffer conditions in the 
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re-multiplexer, etc. Therefore, the controller 1210 manages 
the overall operation of the statistical re -multiplexing. It is 
important here to understand the relationship between the 
process allocator 1204 and the plurality of autonomous 
frame recoders 1206. The process allocator 1204 connects a 5 
given autonomous frame from one video channel to one of 
the recoder resources. The connection pattern is not a static 
one. It is dynamically reconfigured from time to time, 
depending on the overall statistical multiplexing needs and 
rate control operation. 10 

FIG. 13 is a block diagram of a second statistical 
re-multiplexer 1300 using autonomous frame processing 
according to the present invention. The second statistical 
re-multiplexer 1300 is used to process multiple channels, 
and allows the autonomous frame processing to channel hop 35 
between any of the input channels. While FIG. 13 illustrates 
only five channels and a single autonomous frame generator 
and recoder pair, those skilled in the art will recognize that 
this embodiment could be adapted to process any number of 
channels and additional pairs of autonomous frame genera- 20 
tors and recoders could be provided. The channel hopping 
implementation uses the autonomous frame construct to 
recoder multiple channels of video bit streams. Specifically, 
a single recoder can be used to recode the autonomous 
frames from multiple channels of video signals, provided 25 
that the recoder has sufficient processing power to handle the 
tasks. 

The channel hopping implementation of the second sta- 
tistical re-multiplexer 1300 preferably comprises a first 
crossbar switch 1302, an autonomous frame generator 1304, 30 
an autonomous frame recoder 1306, a second crossbar 
switch 1308 and a controller 1310. In FIG. 13, five video 
channels are being recoded by the same autonomous frame 
generator 1304 and autonomous frame recoder 1306. The 
access units of the five channels input to the first crossbar 35 
switch 1302 may or may not be time aligned. The time 
required to process each of the access units usually varies 
from frame to frame. Using the autonomous frame generator 
1304 and the autonomous frame recoder 1306 it is possible 
to recode one access unit in one channel and then recode the 40 
access unit of another channel. The autonomous frame 
generator 1304 and the autonomous frame recoder 1306 can 
either be used to perform the recoding of all channels or 
receding of selected access units of all channels, using the 
approach shown in FIG. 13, The autonomous frame genera- 45 
tor 1304 is of the type described above and converts access 
units into autonomous frames. The autonomous frame 
recoder 1306 is of the type described above and performs 
recoding to produce access units or transport packets con- 
taining those access units. The first crossbar switch 1302 50 
preferably has a plurality of inputs and an plurality of 
outputs. The first crossbar switch 1302 is an input switch 
device that can select the input of the autonomous frame 
generator 1304 to accept access units (or transport packets) 
from any given video channel. Similarly, the second cross- 55 
bar switch 1308 has a plurality of inputs and an plurality of 
outputs, and is an output switch device that is synchronized 
with the first crossbar switch 1302 such that the autonomous 
frame recoder 1306 is always on the same channel as the 
input selected by the autonomous frame generator 1304. In 60 
addition, the first and second crossbar switches 1302, 1308 
have the capability to delineate input signals along the 
access unit boundaries. This is not shown in the figure, but 
should be assumed as a function of first crossbar switch 
1302. The second crossbar switch 1308 performs the reverse 65 
function of the first crossbar switch 1302 correspondingly. 
As shown, this embodiment also includes a plurality of 
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direct transfer paths 1330, 1332, 1334, and 1336 that directly 
transfer the input video channel if the channel is not selected 
for input to the autonomous frame generator 1304. Finally, 
this embodiment includes a controller 1310 coupled to 
control the first and second cross-bar switches 1302, 1308. 
The controller coordinates the operation to make sure that 
the first and second crossbar switches operate in sync, i.e., 
the processed output from chl is sent to the input of crossbar 
switch associated with chl, so on and so forth. 

While the present invention has been described with 
reference to certain preferred embodiments, those skilled in 
the art will recognize that various modifications may be 
provided. These and other variations upon and modifications 
to the preferred embodiments are provided for by the present 
invention which is limited only by the following claims. 
What is claimed is: 

1. A system for transcoding compressed video streams 
using a self-contained data unit processing video data, the 
system comprising: 

a self-contained data unit generator having an input and 
an output for receiving a stream of video data and 
converting the stream of video data into a plurality of 
self-contained data units, the input of the self-contained 
data unit generator coupled to receive compressed 
video streams; and 
a self-contained data unit recoder having an input and an 
output for retrieving recoding parameters, initializing a 
recoder and recoding video data included in the self- 
contained data units, the input of the self-contained 
data unit recoder coupled to the output of the self- 
contained data unit generator. 

2. The system of claim 1 wherein the self-contained data 
unit generator divides the video stream into a plurality of 
self-contained data units, each of the self-contained data 
units having a header portion and a payload portion. 

3. The system of claim 2 wherein each of the self- 
contained data units is an autonomous frame, and an autono- 
mous frame is comprised of: 

a frame header including all header information from the 
original video data plus additional information to allow 
the frame to be recoded, and 
a frame payload including the original video data infor- 
mation. 

4. The system of claim 1, the self-contained data unit 
generator further comprises: 

a header extraction unit having an input and an output for 
extracting header information from the video stream, 
the input of the header extraction unit coupled to 
receive the stream of video data; 
an auxiliary information updating unit having an input 
and an output for generating the recoding parameters, 
the input of the auxiliary information updating unit 
coupled to the output of the header extraction unit; 
a payload transfer unit having an input and an output for 
receiving and identifying picture data and generating a 
frame payload, the input of the payload transfer unit 
coupled to receive the stream of video data; and 
an autonomous frame construction unit having a first 
input, a second input and an output for producing an 
autonomous frame, the first input of the autonomous 
frame construction unit coupled to the output of the 
auxiliary information updating unit to receive the frame 
header, and the second input of the autonomous frame 
construction unit coupled to the output of the payload 
transfer unit for receiving the frame payload. 

5. The system of claim 4, wherein the self-contained data 
unit generator further comprises an access unit generator 
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having an input and an output for receiving packets at the 
transport layer and converting them to access units, the input 
of the access unit generator coupled to receive the stream of 
compressed video data, and the output of the access unit 
generator coupled to the input of the header extraction unit 
and the payload transfer unit. 

6. The system of claim 4, wherein the self-contained data 
unit generator further comprises an autonomous frame 
header buffer having an input and an output for storing to 
store the data necessary for re-encoding any given picture, 
the input of the autonomous frame header buffer coupled to 
the output of the header extraction unit, and the output of the 
autonomous frame header buffer coupled to the input of the 
autonomous frame construction unit. 

7. The system of claim 4, wherein the header extraction 
unit acquires a plurality from the group of picture_header, 
the picture_extension, the extension_user_data, and user_ 
data out of the video data stream. 

8. The system of claim 1, the self-contained data unit 
recoder further comprises: 

a parser having an input and an output for identifying a 
header portion from the self-contained data unit for 
further processing, the input coupled to the output of 
the self-contained data unit generator for receiving 
self-contained data unit, the output of the parser pro- 
viding the identified header portions; 

an initialization unit having an input and an output for 
receiving recoding parameters from the header portions 
identified by the parser and providing the recoding 
parameters as control outputs, the input of the initial- 
ization unit coupled to the output of the parser; 

a recoder unit having an first, second and third inputs and 
an output for modifying the bit rate of a video stream, 
the first input of the recoder coupled to receive picture 
data, the second input of the recoder coupled to the 
output of the initialization unit to receive recoding 
parameters; and 

a controller for controlling the processing of self- 
contained data units, the controller coupled to the 
parser, the initialization unit, and the third input of the 
recoder for controlling the recoding process. 

9. The system of claim 8, wherein the self-contained data 
unit recoder further comprises a pre-parser having an input, 
a first output and a second output for separating the header 
portion and the payload portion of each self-contained data 
unit, the input of the pre-parser coupled to the output of the 
self-contained data unit generator for receiving self- 
contained data units, the first output of the pre-parser 
coupled to the input of the parser, the second output of the 
pre-parser coupled to the first input of the recoder. 

10. The system of claim 8, wherein the self-contained data 
unit recoder further comprises a second parser having an 
input and an output for separating the payload portion of 
each self-contained data unit, the input of the second parser 
coupled to the output of the self-contained data unit gen- 
erator for receiving self-contained data units, the output of 
the second parser coupled to the input of the recoder. 

U. The system of claim 1, wherein the system further 
comprises: 

a second self-contained data unit generator having an 
input and an output for receiving a second stream of 
video data and converting the second stream of video 
data into a plurality of self-contained data units, the 
input of the second self-contained data unit generator 
coupled to receive compressed video streams; and 

a second self-contained data unit recoder having an input 
and an output for retrieving recoding parameters, ini- 
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tializing a recoder and recoding video data, the input of 
the second self-contained data unit recoder coupled to 
the output of the second self-contained data unit gen- 
erator. 

12. The system of claim 11, further comprising: 

a divider having an input and a plurality of outputs, for 
sequences and partitions the stream of video data, the 
input of the divider coupled to receive the stream of 
video data, a first output of the divider coupled to the 
input of the self-contained data unit generator, and a 
second output of the divider coupled to the input of the 
second self-contained data unit generator; and 
a combiner having a plurality of inputs and an output, 
for combining a plurality of bitstreams, the output of 
the divider coupled to provide a stream of video data, 
a first input of the divider coupled to the output of the 
self-contained data unit recoder, and a second input 
of the divider coupled to the output of the second 
self-contained data unit recoder. 

13. The system of claim 12, farther comprising: 

a plurality of additional pairs of a self-contained data unit 
generator and a self-contained data unit recoder serially 
coupled, each of the pair allowing the recoding of video 
data, and 

wherein the divider has a respective output coupled to the 

input of one additional pair; and 
wherein the combiner has a respective input coupled to 

the output of one additional pair. 

14. The system of claim 12, further comprising at least 
one signal line coupled between a respective output of the 
divider and a respective input of the combiner. 

15. The system of claim 1, further comprising: 
a by-pass signal line; 

a first switch coupled to the output of the self-contained . 
data unit generator, the by-pass signal line and the input 
of the self-contained data unit recoder, the first switch 
coupling the output of the self-contained data unit 
generator to either the by-pass signal line or the input 
of the self-contained data unit recoder; and 

a second switch coupled to the output of the self- 
contained data unit recoder, the by-pass signal line and 
an output signal line, the second switch coupling either 
the output of the self-contained data unit recoder or the 
by-pass signal line to the output signal line. 

16. The system of claim 11, further comprising: 

a first cross bar switch having an plurality of inputs and 
a plurality of outputs, for coupling one of the inputs to 
a respective output in response to a first control signal, 
a first output of the first cross bar switch coupled to the 
input of the self-contained data unit generator, and a 
second output of the first cross bar switch coupled to 
the input of the second self-contained data unit gen- 
erator; and 

a second cross bar switch having an plurality of inputs and 
a plurality of outputs, for coupling one of the inputs to 
a respective output in response to a second control 
signal, a first input of the second cross bar switch 
coupled to the output of the self-contained data unit 
recoder, and a second input of the second cross bar 
switch coupled to the output of the second self- 
contained data unit recoder. 

17. A method for processing video data, the method 
including the steps of: 

receiving a video bitstream; 

dividing the video bitstream into a plurality of self- 
contained data units, each self-contained data unit 
including a portion of the video bitstream and recoding 
information; 
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processing the plurality of self-contained data units using 
only information contained therein, wherein processing 
comprises receding video data in the self-contained 
data unit 

18. The method of claim 17 wherein the step of dividing 
the video bitstream into a plurality of self-contained data 
units, further comprises the steps of: 

extracting a plurality of frame headers including header 
information from the original video bitstream and 

extracting a plurality of frame payloads including the 
original video data information, 

19. The method of claim 18 wherein the step of dividing 
the video bitstream into a plurality of self-contained data 
units, further comprises the step of: updating information in 
the plurality of frame headers with the current state of a 
recoder. 

20. The method of claim 19 wherein the step of dividing 
the video bitstream into a plurality of self-contained data 
units, includes the step of temporarily storing the plurality of 
frame headers. 

21. The method of claim 19 wherein the step of dividing 
the video bitstream into a plurality of self-contained data 
units, includes the step of generating a self-contained data 
unit by combining an extracted frame header with updated 
auxiliary information and an extracted payload. 
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22. The method of claim 17 wherein processing the 
plurality of self-contained data units further comprises: 

parsing a self-contained data unit into a header portion 

and a payload portion; and 
initializing receding using the header portion. 

23. The method of claim 22 wherein the step of parsing is 
performed by a first parser that parses a plurality of self- 
contained data units and identifies and outputs the header 
portion of each self-contained data unit, and a second parsers 

1 that parses the plurality of self-contained data units and 
identifies and outputs the payload portion of each self- 
contained data units. 

24. The method of claim 17 wherein at least two of the 
self-contained data units are processed at the same time. 

1 25. The method of claim 24 wherein a first plurality of 
self-contained data units are not processed while the two of 
the self-contained data units arc processed at the same time. 

26. The method of claim 17 wherein at least two of the 
self-contained data units are processed by different autono- 

1 mous recoding units. 

27. The method of claim 17 wherein a first self-contained 
data unit is processed and a second self-contained data unit 
is not processed. 

***** 
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